fix: improve state serialization #6563
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## unstable #6563 +/- ##
============================================
+ Coverage 61.49% 61.53% +0.03%
============================================
Files 556 557 +1
Lines 58895 58954 +59
Branches 1856 1859 +3
============================================
+ Hits 36216 36275 +59
Misses 22638 22638
Partials 41 41 |
jeluard
reviewed
Mar 19, 2024
jeluard
reviewed
Mar 19, 2024
| validator: ValueOfFields<typeof ValidatorType> | ||
| ): number { | ||
| output.set(validator.pubkey, offset); | ||
| offset += 48; |
There was a problem hiding this comment.
Is there a way we could document those constants? Or maybe assign them to variable so that they are more self-explanatory?
Performance Report✔️ no performance regression detected Full benchmark results
|
twoeths
commented
Mar 25, 2024
| } | ||
| } | ||
|
|
||
| function writeEpochInf(dataView: DataView, offset: number, value: number): number { |
There was a problem hiding this comment.
nflaig
reviewed
Mar 25, 2024
packages/state-transition/test/perf/util/serializeState.test.ts
Outdated
Show resolved
Hide resolved
twoeths
force-pushed
the
tuyen/validators_type
branch
from
3ee006b to
121edfe
Compare
nflaig
reviewed
Mar 26, 2024
nflaig
approved these changes
Mar 27, 2024
| name: "lodestar_cp_state_cache_state_persist_seconds", | ||
| help: "Histogram of time to persist state to db", | ||
| stateSerializeDuration: register.histogram({ | ||
| name: "lodestar_cp_state_cache_state_serialize_seconds", |
There was a problem hiding this comment.
There are few other metrics we could consider renaming, those are still unused (not part of dashboard)
But rather part of another PR
There was a problem hiding this comment.
yes will persist all metrics in a follow-up PR
|
🎉 This PR is included in v1.18.0 🎉 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
State serialization happens once per epoch in n-historical state so its performance is important
Description
value_serializeToBytesperformance, this helps makestate.validators.serialize()3.4x fasterbefore
after
serializeToBytes()api of ssz v0.15.1part of #5968